Context

Multiple Zendesk tickets related to whitehall assets being live when they should not be, triggered an analysis of the problematic data - PR. We've investigated the assets further, extracting a batch of assets that should not be live, because they should be replaced.

A subset of these assets correspond to a now identified bug where if the user replaces an attachment on a new draft, without having saved the edition draft first, the supposedly replaced original asset remains live, because it fails to redirect to its replacement. The data extraction rake task looped through all AttachmentData's where there deleted? method is false, that had a replacement set in whitehall, but no corresponding replacement value set on their assets in asset-manager. The resulting subset of assets that have this issue can be found here.

This PR provides a rake task to address the assets that are likely a result of this bug, by populating their replacement asset in asset-manager. It takes whitehall as a source of truth, relying on the AttachmentData and Asset objects to deduce the value of the replacement_id field.

This is only a data patch solution. A fix for the issue will be implemented separately.

The rake task

This task expects a csv file containing, on each row, the AttachmentData ID, the Asset ID (from asset-manager), and the asset's variant (i.e., "original" or "thumbnail").

We originally tried to run the the AssetManager::AttachmentUpdater.replace(replaced_attachment_data) line on a selection of AttachmentData IDs, but that only linked up the corresponding assets to their immediate successors, which were likely in draft. There is some logic in asset-manager that ensures all assets in the chain point to the last one, after a publish event. The data we're working with is now superseded and it's difficult to put it back to normal using the common flows of the code, so we replicated that logic for the rake task:

• If the AttachmentData does not have a replacement, we skip. The reported asset IDs we're running this on were extracted based on whether a replaced_by_id field was set. Nonetheless, some of those AttachmentDatas are deleted.
• We identify the last AttachmentData in the chain.
• We figure out the corresponding asset variant we need to run an update call on (based on a variant value passed through in the csv).
• We update asset-manager, logging any errors.

We expect the end of the line replacement to either still be live, or a broken asset (for example a draft + deleted one, which corresponds to another bug we know of) that will be taken care of in the next rake task. We will be running this rake task first, to ensure we have a healthy replacement chain, then catch all other issue with the next rake task. There are a few errors in the trial run logs, including unprocessable content and parent document url validation, but we expect any problematic data to be caught by the next rake task; the few not found errors are assets that have already been deleted (perhaps manual intervention), which means they are not a problem anymore.

Links

Trello card
Assets we fixed
Data extraction/reporting rake task
Assets investigation document

Co-authored-by @minhngocd